Algorithmic Differentiation of Numerical Methods: Tangent-Linear and Adjoint Solvers for Systems of Nonlinear Equations

We discuss software tool support for the Algorithmic Differentiation (also known as Automatic Differentiation; AD) of numerical simulation programs that contain calls to solvers for parameterized systems of n nonlinear equations. The local computational overhead as well as the additional memory requirement for the computation of directional derivatives or adjoints of the solution of the nonlinear system with respect to the parameters can quickly become prohibitive for large values of n. Both are reduced drastically by analytical (in the following also: continuous) approaches to differentiation of the underlying numerical methods. Following the discussion of the proposed terminology we develop the algorithmic formalism building on prior work by other colleagues and we present an implementation based on the AD software dco/c++. A representative case study supports the theoretically obtained computational complexity results with practical run time measurements.